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ABSTRACT 

Sixty Naval Postgraduate School students served in a 
verbal discrimination (VD) experiment with 2-, 3-, and 
4-word items and presentation rates per item of 1.5 or 3.0 
seconds. Half the items had similar and half, dissimilar 
words. Based on information theory, lists of different 
lengths were prepared for 2-, 3-, and 4-word items. The 
lists were equated for overall load at 20 bits of infor- 
mation. Performance was consistent with the equal-load 
hypothesis and a differential of two in the amount of 
information transferred was observed because of the rate 
factor. Analysis of variance of correct responses revealed 
significant effects for item length, presentation time, 
similarity, and trials. 
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I. 



INTRODUCTION 



Underwood and Freund (1969) have shown that learning 
difficulty varies as the number of choices in a verbal 
discrimination (VD) item are varied when the total number 
of items in a list are held constant. Zacks (1969) has 
demonstrated that the total learning time tends to be 
invariant over various conditions of practice for a fixed 
task load. These characteristics of VD learning tasks 
would seem to have much in common with the information 
analysis of communication tasks (Garner 1962). That is, 
the number of possible alternatives in any VD item could 
be readily quantified in terms of the amount of information 
contained in the item if the expected relative frequency 
(a priori probability) of choice of each word could be 
identified. The total amount of information in a list 
would then be the sum of the information contained in each 
item of the list. Similarly, the invariance of total 
learning time could be expressed quantitatively in infor- 
mation measures as the rate of information processing. 

The information analysis of VD tasks with more than two 
items might also provide additional information over con- 
ventional methods of analyzing VD learning. Conventional 
methods quite often count the number of correct and 
incorrect responses and analyze them separately. Informa- 
tion measures, when the responses are summed over 
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individuals , provide information regarding the patterning 
of choices over all the alternatives considered simul- 
taneously. That is, learning can be expressed as a 
reduction in uncertainty demonstrated by the deviation of 
choices from a pattern of random choices , which has the 
maximum uncertainty. 

To present the VD learning in the framework of informa- 
tion theory requires a method of reinforcement that provides 
a constant amount of information over all tasks. If the 
conventional noncontingent reinforcement procedures are 
used, the designation of a correct response from a 2-word 
item would provide one bit of information, but the rein- 
forcement provided by the designation of a correct response 
from a 4-word item would present two bits of information. 

This problem could be alleviated for the 4-word item, for 
example, by presenting the four words first and then 
presenting only one of the words to the subject and asking 
the subject whether it is the correct response. Another 
problem is created by this solution, however, since the 
subject can optimize his chances of being correct by always 
responding "no." The simple solution, therefore is con- 
tingent reinforcement using the anticipation method. In 
this procedure, the subject is presented the alternatives 
and he is positively reinforced only when he chooses the 
correct alternative. 

To summarize, VD learning could be analyzed using 
procedures from information theory if the initial probability 
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of choice of each word in a VD item is known and only con- 
tingent reinforcement of correct responses is provided. 
Learning could then be analyzed as the reduction in 
uncertainty of the subjects' responses from the uncertainty 
initially present in the item. Ideally, it would be 
desireable to have all the choices in any item equally 
probable. In this case, the information content is defined 
simply 

I = log 2 N, 

where N equals the number of words. A 2-word item would 
have one bit of information; a 3-word item, 1.585 bits; and 
a 4-word item, two bits of information. 

Using the foregoing notions, this study will examine the 
learning of 2-, 3-, and 4-word VD items using two presenta- 
tion rates. In addition, the effects of similarity of the 
words in an item will also be examined. 
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II. METHOD 



A. WORD LISTS 

Three word lists, one each for the 2-, 3-, and 4-word 
treatments, were constructed and constrained to a maximum 
uncertainty of 20 bits.'*' For the uncertainty level selected, 
lists of 10, 12, and 20 discrimination items were required 
for the 2-, 3-, and 4-word treatments, respectively. The 
lists are shown in Table I. In order for the words in each 
item to have an equal a priori probability of selection 
on the first trial, three criteria for word selection were 
used. First, only words that were considered to have a 
high background frequency for all subjects were used. These 
words were selected from categories having at least a .9 
correlation over test subjects in the category norms for 
verbal items compiled by Battig and Montague (1969). 

Secondly, half of the items in each list was constructed 
from words from the same verbal category by using words 
with as close to the same frequency as possible. The 
remaining items used words matched by the same response 
frequency from two, three, or four different categories, 
depending upon the respective discrimination tasks. Finally, 
all words in a discrimination task were required to have 

■*"In order to maintain list compatability for similar and 
dissimilar words, the three word discrimination list con- 
tained 19.02 bits of uncertainty. 
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TABLE I 



WORD LISTS 


FOR 2-, 3-, 


AND 4 -WORD 


TREATMENTS 






Two-Word 


Treatments 


• 


murder 


wine* 


juice 


doll* 


apple 


river* 


book 


lake* 


iron 


yard* 


nail 


swim* 


tea 


coffee 


temple 


rock* 


table 


chair 


cotton 


salt* 


mother 


father 


bus 


gun* 


cat 


dog 


water 


door* 


eye 


head 


car 


train 


foot 


mile 


red 


blue 


corn 


bean 


hour 


minute 





Three-Word Treatment 




door 


temple 


water* 


brother 


mother 


father 


green 


blue 


red 


lake 


book 


table* 


cotton 


hours 


salt* 


apple 


knife 


cat* 


iron 


yard 


doctor* 


eye 


head 


foot 


minute 


hour 


second 


hill 


river 


rock 


boat 


train 


car 


nail 


oil 


swim* 





Four-Word 


Treatment 




mother 


sister 


brother 


father 


door 


private 


temple 


water* 


eye 


foot 


nose 


head 


swim 


nail 


wine 


oil* 


yard 


doctor 


iron 


book* 


yellow 


blue 


green 


red 


lake 


rock 


river 


hill 


cotton 


salt 


house 


table* 


cat 


murder 


knife 


apple* 


year 


minute 


hour 


second 



Denotes dissimilar word groups. First word in each item 
was used as the correct response word in the experiment. 
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essentially the same frequency in the Thorndike and Lorge 
(1944) general count. In this respect, nearly all words 
used had AA or A word frequencies. Thus, each item had 
words of approximately equal response frequency in the 
Battig and Montague norms and all words had relatively high 
frequencies of use in the English language. The latter 
characteristic is important from the point of view of 
frequency theory (Ekstrand, Wallace, and Underwood, 1966), 
which states that discrimination learning of frequent words 
is more difficult because of their high background exposure 
prior to the subject's participation in a VD experiment. 

The correct word within any item was designated by random 
selection. 

Using as a basis for decision the results of Underwood 
and Freund's (1970) work in VD retention, it was felt that 
six presentations of each list would be of sufficient length 
to observe trends in the results. The order of items in 
each of the six trials was randomized, as was the word order 
within each item. 

B. DESIGN 

Each discrimination task was considered to form a 1 X 2, 
1X3, or 1 X 4 matrix having a uniform a priori probability 
distribution for the first selection of any word in the 
matrix. All responses, right or wrong, were recorded and 
used to form the a posteriori distribution over the alter- 
natives. Further, the experiment conformed to a 2 X 3 X 2 
mixed factorial with two levels of presentation time (1.5 and 
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3 sec.)* The three different item lengths, as described, 
were presented as a between subject variable, while the 
similar or dissimilar word groupings were considered as a 
within subject variable. Treating the similar and dis- 
similar word groupings as fixed on a per trial basis, 
resulted ina2X3X2X6 fixed factorial for analysis 
of the trials or trends variance. 

C. SUBJECTS 

The 60 subjects used were graduate level students in 
the operations research curriculum at the Naval Post- 
graduate School. They were volunteers and randomly assigned 
to the six treatment groups. 

D. PROCEDURES 

All subjects, following an explanation of the subject's 
task and procedures, were individually run by presenting 
the word lists on a Lafayette high-speed memory drum. The 
discrimination item was presented for a presentation time 
of 1.5 or 3.0 seconds. A blank space appeared for a similar 
interval for the interitem interval. The subject announced 
the word he believed to be the right discrimination and was 
reinforced with the verbal response "correct" from the 
experimenter if the right word was chosen, otherwise, there 
was no response. No additional time was given between trials 
i.e., the six trials were run without a break. 
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III. RESULTS 



The results will first be analyzed according to the 
growth of correct responses, then an analysis in terms of 
information transfer findings will be made. The percent 
of correct responses per trial by item length and presen- 
tation rate is shown in Table II. A graphic presentation 
of these data are shown in Figure 1; The percent of 
correct responses per trial as a function of similarity is 
shown in Tables III and IV, and a graph of the 3-second 
presentation rate treatments is shown in Figure 2. 



TABLE II 

PERCENT OF CORRECT RESPONSES PER TRIAL 
BY ITEM LENGTH AND PRESENTATION RATE 

4-Words 3-Words 2-Words 



1.5-Sec. 3-Sec. 1.5-Sec. 3-Sec. 1.5-Sec. 3-Sec. 



1 


29.0 


30.0 


33.3 


34.1 


48.5 


46.5 


2 


28.0 


41.0 


43.3 


46.7 


54.0 


64.5 


3 


32.0 


42.0 


41.7 


65.0 


55.0 


76.0 


4 


34.0 


55.0 


55.0 


65.0 


64.5 


79.0 


5 


42.0 


62.0 


50.8 


73.3 


66.0 


83.5 


6 


40.0 


69.0 


60.0 


82.5 


76.0 


87.5 
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Figure 1. Percent Correct Responses for Each Treatment. 



•OOT 



*0 'd 
U M 

0 O 

s & 

1 I 

CM 00 



'd *d *d 
M M M 

0 o o 
S B S 

1 t i 
cm oo 



*0 

U 

i 

l 




u 

fd 



•H 

e 

•H 

CO 



d 



•H 

e 

•H 

W 

W 

•H 

Q 




VO 



in 



W 

i— I 

fd 

*H 

M 

Eh 



— 00 




r 



T - 

LD 



I 

O 

in 

^091103 ^ueojsd 



in 

CM 



o 



15 



Figure 2. Percent Correct Responses for Similar and Dissimilar 
Words Over 3-Second Presentation Rate. 



TABLE III 



PERCENT OF CORRECT RESPONSES 
FOR SIMILAR WORDS 



4-Words 3-Words 2-Words 



1.5-Sec. 3-Sec. 1.5-Sec. 3-Sec. 1.5-Sec. 3-Sec. 



1 


32.0 


34.0 


33.3 


36.7 


48.0 


52.0 


2 


26.0 


42.0 


45.0 


48.3 


55.0 


68.0 


3 


30.0 


46.0 


43.3 


70.0 


57.0 


73.0 


4 


44.0 


60.0 


55.0 


73.3 


67.0 


70.0 


5 


46.0 


70.0 


58.3 


81.7 


60.0 


80.0 


6 


48.0 


78.0 


65.0 


88.3 


71.0 


83.0 



-TABLE IV 

PERCENT OF CORRECT RESPONSES 
FOR DISSIMILAR WORDS 



4-Words 3-Words 2-Words 

Trial 

1.5-Sec. 3-Sec. 1.5-Sec. 3-Sec. 1.5-Sec. 3-Sec . 



1 


26.0 


26.0 


33.3 


31.5 


49.0 


41.0 


2 


30.0 


40.0 


41.6 


45.1 


53.0 


61.0 


3 


34.0 


38.0 


40.1 


60.0 


53.0 


79.0 


4 


26.0 


50.0 


55.0 


56.7 


62.0 


88.0 


5 


38.0 


54.0 


43.3 


64.9 


72.0 


87.0 


6 


32.0 


60.0 


55.0 


76.7 


81.0 


92.0 
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The data were analyzed first using an analysis of 
variance over all subjects and all trials. Accordingly, 
item length and presentation rate were between subjects 
treatments and similarity was a within subject treatment. 
The results are shown in Table V. The main effects were 
all statistically significant, item length at less than 
the .001 level of probability and the similarity effect 
at less than .01. None of the interactions approached 
statistical significance. 



TABLE V 

ANALYSIS OF VARIANCE OVER 
SUBJECTS FOR ALL TRIALS 



Source 




df 


MS 


F 


Between Subjects 




(59) 






Word List 


(A) 


2 


6213.02 


31.92** 


Rate 


(B) 


1 


4953.68 


25.45** 


A X B 




2 


67.72 


— 


Error 




54 


194.60 




Within Subjects 




(60) 






Word Similarity (C) 


1 


1122.41 


7.96* 


A X C 




2 


93.26 


— 


B X C 




1 


7.00 


— 


A X B X C 




2 


198.27 


— 


Error 




54 


140.95 




Total 




119 







*P < .01 

**P < .001 
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In order to analyze trend effects across trials, an 
analysis of variance was conducted using nonrepeated 
measures. The basic datum for this analysis was the percent 
correct for each of 12 item length (3) x presentation rate 
(2) x similarity (2) treatments for each of six trials 
making a total of 71 degrees of freedom. The results of 
the analysis are shown in Table VI. None of the triple 



TABLE VI . 

ANALYSIS OF VARIANCE OVER TRIALS 





Source 


df 


MS 


F 


Word List (A) 


2 


3650.96 


183.36** 


Rate 




(B) 


1 


3430.68 


172.31** 


Similarity (C) 


1 


369.01 


18.53* 


Trials 


(D) 


5 


1655. 84 


66.31*** 


A 


X 


B 




2 


16.68 




A 


X 


C 




2 


244.35 


4.45* 


A 


X 


D 




10 


26.74 




B 


X 


C 




1 


26.65 




B 


X 


D 




5 


195.29 


7.82** 


C 


X 


D 




5 


8.88 




A 


X 


B 


x c 1 


2 


19.91 




A 


X 


B 


X D 2 


10 


34.21 




A 


X 


C 


X D 2 


10 


54.94 




B 


X 


C 


X D 2 


5 


11.50 




A 


X 


B 


X C X D 3 


10 


24.97 




Total 






71 







1. Error term of A, B, C. 



2. Error terms for corresponding first order (2-way) inter- 
actions not involving D. 

3. Error term for trials, the second order (3-way) inter- 
actions and the first order (2-way) interactions involv- 
ing D. 

*p < .05, **p < .01, ***p < .001 
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interactions was significant when compared with the 4-way 
interaction. Accordingly, each was used as the error term 
as shown in the table for its corresponding main effects 
(less trials) and the 2-way interactions (less trials). The 
4-way interaction was used as the error term for trials and 
the 2-way interactions involving trials. The results of the 
analysis show the effects for list length, presentation rate, 
and trials significant with a probability less than .01. 

The similarity effect was just significant at p = .05. Of 
the interactions, only the list length x similarity (p .05) 
and the rate x trials effect (p <. .01) reached statistical 
significance . 

The analyses confirm what is evident in the performance 
curves of Figure 1. The amount learned is clearly dependent 
on the rate of presentation and, within each rate of 
presentation, the number of words in each item (item length). 
The significant rate by trials interaction confirms the 
apparent differences in the slope of the curves for presen- 
tation rate in Figure 1. The rate of learning is greater 
for the slower presentation rate. The reason why the 
similarity main effect is not clearly established in the 
trials analysis is given in the significant interaction of 
similarity by list length. It appears that similarity 
is a facilitative factor for three and four word items , but 
does not have this effect in the 2-word items. 

The first step in the information transfer analysis was 
to compute the relative frequency that each choice was 
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selected on its first presentation. This was done as a 
check on the original assumption that each alternative or 
choice was equally likely to be chosen on the first trial. 
For the 2-word items, these probabilities can be obtained 
from Table II. For the 3-, and 4-word items, each word 
within an item was arbitrarily designated as word 1, 2, 3, 
or 4 (depending on the number of alternatives) . The a 
posteriori probability for each classification is presented 
in Table VII for the 3-, and 4-word' lists . Chi-square tests 
indicated that the null hypothesis of equal (random) choice 
of the alternatives could not be rejected (p > .2). Thus, 
the assumed prior distribution is upheld by the empirical 
results (posterior distribution). To recapitulate, each 
2-word item had one bit of uncertainty; each 3-word item, 
1.585 bits of uncertainty; and each 4-word item, two bits 
of uncertainty. The 2-word and 4-word lists had 20 bits of 
uncertainty, and the 3-word list, 19.02. 

To compute information transfer, the next step was to 
determine the distribution of choices for each item on each 



TABLE VII 

POSTERIOR DISTRIBUTION OVER WORD LISTS 
OF 3-, 4 -WORD ITEMS 







Word 1 


Word 2 


Word 3 


Word 4 


Chi-Square 


4-Word 


Items 


28.0 


27.0 


21.0 


24.0 


1.20 


3-Word 


Items 


33.3 


36.0 


30.4 


— 


.47 
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trial. Once this was accomplished, it was then a simple 
matter to obtain the uncertainty for the item set using 
tables of 

U (x,y ) = -Z i p ± log p i# 

where the logarithm is to the base two. It is obvious that 
when perfect learning occurs, p^ for the correct choice is 
1.00 and the probabilities for the other choices are zero. 

The logarithm of 1.00 is zero and there is no longer any 
uncertainty. Thus, U(x,y), the uncertainty at each trial 
is an estimate of how much is yet to be learned (or trans- 
ferred) . The original uncertainty less U(x,y) is the 
amount of information transferred. It is evident, therefore, 
that each treatment condition had the same amount of infor- 
mation to transfer initially. Furthermore, those subjects 
in the 1.5-second presentation rate were being asked to 
transfer the information at a rate twice as high as the 
subjects in the three-second presentation rate. The U(x,y) 
values for each item length by presentation rate treatment 
condition is shown in Table VIII by trials. The data are 
presented graphically in Figure 3. 

The trends are quite clear that there is no large 
difference in the amount of information processed by any 
item-length condition within each presentation rate. It 
appears, however, that the 4-item, 1.5-second rate was at 
or close to the channel capacity, since negligible amounts 
of information were transferred. The uncertainty remaining 
at trial six, averaged over the 1.5-seconds presentation 
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Figure 3. Uncertainty Remaining per Trial for Each 
Presentation Rate by Item Length. 



TABLE VIII 



UNCERTAINTY REMAINING PER PRESENTATION 
RATE OVER TRIALS 



4-Words 3-Words 2-Words 

Trial 

1.5-Sec. 3-Sec. 1.5-Sec. 3-Sec. 1.5-Sec. 3-Sec. 



1 


16.9 


17.0 


18.2 


17.5 


18.2 


18.4 


2 


18.0 


17. 8 


16.9 


16.3 


18.6 


17.4 


3 


17.3 


16.9 


16.5 


13.3 


18.4 


13.0 


4 


17.1 


14.1 


15.6 


13.2 


15.9 


12.6 


5 


17.1 


13.5 


15.6 


11.5 


16.0 


10.7 


6 


16.6 


10.3 


15.0 


7.1 


14.0 


8.2 



rate treatments was 15.2 bits and the amount remaining for 
the three-seconds treatments was 8.4 bits. Thus, it appears 
that the rate differential demanded by the experimental 
conditions is quite clearly apparent in the results. It 
should be noted that there is considerable information yet 
to be transferred at the end of six trials. 

A comparison of the percent correct and information 
transferred measures is presented in Figure 4 for the 
3-seconds presentation rate by item-length. The differences 
in elevation of the curves within the percent correct curves 
is, as would be expected, much greater and consistent than 
in the information transfer curves. The percent correct 
curves show (albeit not clearly) the monotonically rising, 
negatively accelerated curve typical of learning curves. 
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Figure 4. Information Gained per Trial Versus Percent Correct 
per Trial for 3-Second Presentations. 



There is a trend in the information transfer curve for 
positive acceleration. The reason for the difference is 
evident in the fact that, with learning reaching the 75 
percent correct point (and beginning to be constrained by 
the 100 percent ceiling) , the information transferred has 
just passed the 50 percent mark of the total information 
to be transferred. 
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IV. DISCUSSION 



This experiment has demonstrated that it is possible to 
quantify the difficulty of a VD item. The validity of the 
quantificaticn was demonstrated by creating lists of equal 
workload but different in lengths and containing items 
differing in difficulty. The results were generally con- 
sistent with the equal workload quantifications. Moreover, 
the lists were processed at two rates, one twice as fast 
as the other, making the work/rate condition twice as high 
for the groups with short processing times. Again, the 
results showed that the differential in work output (infor- 
mation transferred) was close to a factor of two. Finally, 
it was suggested that an information transfer analysis of 
VD learning might provide a more sensitive measure than the 
percent of correct responses because it approaches the 
ceiling in a slower, positively accelerated manner. 

The results of the experiment are also consistent with 
expectations from frequency theory (Ekstrand, Wallace, and 
Underwood, 1966), in the following ways. First, it can be 
assumed that individuals in the 3-second presentation rate, 
who had 3-second response (and rehearsal) periods, had a 
greater opportunity to rehearse the correct response (RCR) , 
once it had been identified. It should be mentioned that 
the 2-word groups had an advantage in this respect since an 
initial right or wrong (unreinforced) answer served to 
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identify the correct response. This may account for the 
slight superiority of the 2-word groups in the experiment. 

On the other hand, it should also be noted that these are 
assumptions and that there was no way within the design to 
determine to what extent RCRs were made by the subjects. 

Closely associated with the foregoing observation is 
another based on frequency theory regarding the differential 
difficulty in learning the 2-, 3-, and 4-word items. Fre- 
quency theory would state that it was more difficult to 
establish a differential frequency for the correct response 
in those items with a greater number of choices. In the 
case of the 4-word items, for example, there would be no 
opportunity to establish a differential frequency for the 
correct response if a subject made three incorrect responses 
to the same item or two incorrect responses to two different 
items in the first five trials. Frequency theory, however, 
does not provide a quantitative measure of the relative 
difficulty of n-word items. The information value of a 
set of n-words in an item does provide a measure of this 
difficulty based on a similar concept of the probability 
of choice of any one word within the set. That is, the 
difficulty in establishing a relative frequency advantage 
for any one alternative is directly related to the infor- 
mation content of the item. For example, a 3-word item 
with one highly predominant choice (high background frequency) 
would not have as high an information value as a 3-word item 
in which the choices are apparently all equal (as in this 
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study). Accordingly, it is suggested that a fruitful arena 
of future research would be to attempt to analyze informa- 
tion transmission with items having unequal a priori 
distribution of choices. 



28 



APPENDIX A 



SUBJECT'S INSTRUCTIONS 

You are participating in a verbal discrimination experi- 
ment. You will be shown one series of either 2-, 3-, or 
4-word items, one of which has been arbitrarily selected as 
correct. The list of words is 10, 12, or 20 items long and 
will be repeated in various orders for six trials. You will 
have 1. 5/3.0 seconds to view the words. Subsequent to each 
word group, there will be a blank space of 1. 5/3.0 seconds 
duration. It is your task to view the words and guess which 
one is correct. Once you have selected your word, announce 
it to the experimenter. If your response is correct, the 
experimenter will tell you that you are correct, otherwise 
no answer will be given your response. In each item, the 
correct response word will remain the same throughout the 
experiment. 
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APPENDIX C 



PERCENT CORRECT RESPONSES FOR CORRECT RESPONSE WORD 
(POSTERIOR DISTRIBUTION) 



Response Word* 
mother 


1 


2 


3 


4 


5 


6 


10 


00 


10 


60 


50 


70 


door 


00 


30 


00 


50 


30 


20 


eye 


30 


40 


20 


40 


40 


30 


swim 


50 


10 


60 


20 


50 


30 


yard 


40 


40 


30 


20 


10 


40 


yellow 


40 


30 


50 


40 


60 


70 


lake 


50 


30 


50 


50 


50 


30 


cotton 


10 


20 


30 


10 


40 


20 


cat 


30 


50 


40 


30 


60 


50 


year 


30 


30 


20 


20 


30 


40 








1.5-SECOND 






door 


50 


30 


60 


70 


30 


70 


cotton 


30 


20 


40 


40 


70 


50 


cat 


30 


60 


30 


50 


60 


80 


swim 


10 


60 


40 


40 


.50 


40 


yellow 


30 


40 


50 


80 


90 


90 


lake 


50 


50 


50 


70 


70 


70 


mother 


00 


30 


20 


40 


60 


90 


eye 


30 


30 


60 


50 


70 


60 


yard 


10 


30 


20 


60 


60 


70 


year 


40 


60 


40 


60 


70 


80 



3-SECOND 



* 

Four-word treatment over both presentation rates. 



36 



Trials 
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